Action Planning System and Method for Autonomous Vehicles

ABSTRACT

An action planning system ( 100 ) and method for autonomous vehicles are provided. The system ( 100 ) comprises one or more processors ( 108 ) and one or more non-transitory computer-readable storage medium ( 110 ) having stored thereon a computer program used by the one or more processors ( 108 ), wherein the computer program causes the one or more processors ( 108 ) to estimate future environment of an autonomous vehicle ( 114 ), generate a possible trajectory for the autonomous vehicle ( 114 ), predict motion and reactions of each dynamic obstacle in the future environment of the autonomous vehicle ( 114 ) based on current local traffic context, and generate a prediction iteratively over timesteps.

FIELD

This disclosure relates generally to autonomous vehicles and, moreparticularly, to action planning systems and methods for autonomousvehicles.

BACKGROUND

Unless otherwise indicated herein, the materials described in thissection are not prior art to the claims in this application and are notadmitted to the prior art by inclusion in this section.

Conventional action planning device is based on a finite state machine(FSM) approach where the autonomous vehicle decides to make specificactions based on heuristic transition conditions. This FSM approach,however, only allows the autonomous vehicle to passively react tochanges in its environment and provides no future look ahead for how thegiven action might affect how the future traffic situation evolves.FIGS. 1 and 2 illustrate various possible actions such as “follow lane”action with a target or “lane change” action between two target vehicleson an adjacent lane that are evaluated by the prior art action planningdevices using the FSM approach.

SUMMARY

A summary of certain embodiments disclosed herein is set forth below. Itshould be understood that these aspects are presented merely to providethe reader with a brief summary of these certain embodiments and thatthese aspects are not intended to limit the scope of this disclosure.Indeed, this disclosure may encompass a variety of aspects that may notbe set forth below.

Embodiments of the disclosure related a non-transitory computer-readablestorage medium having stored thereon a computer program for evaluatingreactions and interactions with one or more vehicles, the computerprogram comprising a routine of set instructions for causing the machineto perform searching through a tree of possible action sequencecombinations for an ego vehicle and capturing reactions and interactionswith one or more vehicles.

Another aspect of the disclosed embodiment is a method by one or moreprocessors includes estimating future environment of an autonomousvehicle, generating a possible trajectory for the autonomous vehicles,predicting motion and reactions of each dynamic obstacle in the futureenvironment of the autonomous vehicle based on current local trafficcontext, and generating a prediction iteratively over timesteps. Themethod further decoupling, by the one or more processors, an actiondecision time resolution from an iterative prediction resolution.

Another aspect of the disclosed embodiment is a system for an autonomousvehicle includes one or more processors and one or more non-transitorycomputer-readable storage medium having stored thereon a computerprogram used by the one or more processors, wherein the computer programcauses the one or more processors to estimate future environment of anautonomous vehicle, generate a possible trajectory for the autonomousvehicles, predict motion and reactions of each dynamic obstacle in thefuture environment of the autonomous vehicle based on current localtraffic context, and generate a prediction iteratively over timesteps.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features, aspects, and advantages of this disclosurewill become better understood when the following detailed description ofcertain exemplary embodiments is read with reference to the accompanyingdrawings in which like characters represent like arts throughout thedrawings, wherein:

FIG. 1 is a simplified diagram showing a possible follow-lane actionusing a prior art FSM implemented action planning device for anautonomous vehicle;

FIG. 2 is a simplified diagram showing a possible lane-change actionusing the prior art FCM implemented action planning device for anautonomous vehicle;

FIG. 3A is an illustration showing an automated driving system accordingto an embodiment of the disclosure;

FIG. 3B is a simplified diagram showing a bird eye view of a map and amultilane road including a plurality of neighboring vehicles inproximity to an autonomous vehicle according to a described embodimentof the disclosure; and

FIGS. 4A-4D are views of the autonomous vehicle traveling autonomouslyin proximity to a plurality of neighboring vehicles according to adescribed embodiment of the disclosure.

DETAILED DESCRIPTION

The following description is presented to enable any person skilled inthe art to make and use the described embodiments, and is provided inthe context of a particular application and its requirements. Variousmodifications to the described embodiments will be readily apparent tothose skilled in the art, and the general principles defined herein maybe applied to other embodiments and applications without departing fromthe spirit and scope of the described embodiments. Thus, the describedembodiments are not limited to the embodiments shown, but are to beaccorded the widest scope consistent with the principles and featuresdisclosed herein.

FIGS. 3A and 3B illustrate an automated driving system 100 in accordancewith one aspect of the disclosure. As depicted in FIG. 3A, the drivingsystem 100 may be either integrated into a vehicle 114, a machinedevice, or any suitable portable or mobile device/vessel. While certainaspects of the disclosure are particularly useful in connection withspecific types of vehicles, the vehicle may be any type of vehiclesincluding, but not limited to, cars, trucks, motorcycles, buses, boats,sport-utility vehicles, two-wheelers, airplanes, helicopters,lawnmowers, recreational vehicles, amusement park vehicles, trams, golfcarts, trains, trolleys, ultralights, and the like. The vehicle may haveone or more automated driving systems. The machine devices may be anytype of devices including, but not limited to, cellular phones, laptops,tablets, wearable devices such as watches, glasses, goggles, or anysuitable portable devices. As depicted in FIG. 3A, the automated drivingsystem 100 includes a processor 108, a computer readable medium 110, anda communication module 112. A route planning module 102, an actionplanning module 104, and a trajectory planning module 106 may beprovided in the automated driving system 100.

Although the route planning module 102, the action planning module 104,the trajectory planning module 106, the processor 108, the computerreadable medium 110, and the communication module 112 as being withinthe same block 100, it will be understood by those of ordinary skill inthe art that the route planning module 102, the action planning module104, the trajectory planning module 106, the processor 108, the computerreadable medium 110, and the communication module 112 may or may not behoused in the same block 100. In various of the aspects describedherein, the processor 108, the computer readable medium 110, and thecommunication module 112 may be integrated into a computer locatedoutside the automated driving system 100. In other aspects, some of theprocesses described herein are executed on a processor disposed withinthe vehicle 114 and others by a remote processor disposed within themachine device 116.

The computer readable medium 110 stores information accessible by theroute planning module 102, the action planning module 104, thetrajectory planning module 106, and the processor 108 includingcomputer-executable instructions that may be executed or otherwise usedby route planning module 102, the action planning module 104, thetrajectory planning module 106, and the processor 108. The processor 108may be any conventional processor. Alternatively, the processor 108 maybe a dedicated device such as an ASIC.

The communication module 112 in wired/wireless communication with othercomputers such as one or more electronic controller units, one or moreprocessors, networks, directly or indirectly. Other sensing devices suchas sensors, user interface such as a touch display, mouse, keyboard,audio input, camera, and other suitable computer implemented modules maybe integrated into or communicatively coupled to the system 100.

Processes of route, action, and trajectory running on the route planningmodule 102, the action planning module 104, and the trajectory planningmodule 106 generate candidate route, action, and trajectory that an egovehicle may follow through an environment during a configurable timehorizon T. In some aspects, processes of route, action, and trajectoryto generate candidate route, action, and trajectory may be run on theprocessor 108 whereby the route planning module 102, the action planningmodule 104, and the trajectory planning module 106 arecomputer-executable instructions being programmed on the processor 108.

As depicted in FIG. 3B, the action planning module 104 is operable toperform a limited horizon search through a tree of possible actionsequence combinations for the autonomous vehicle and to combine thehorizon search with iterative, full-environment prediction to capturereactions and interactions with other traffic participants. For example,the action planning module 104 performs an optimal graph search toefficiently explore the tree of possible ego actions up to a limitedtime horizon or a planning horizon. The time horizon includes a startpoint of time and an end point of time at which the time horizon can beevaluated. The time horizon is necessary to constrain the size of thesearch space and enable efficient re-planning.

In order to ensure the end point of time is quickly reached, whereby themost suitable path can be found without having to traverse all possiblepaths or connections, a graphic search technique is used. The graphsearch technique may be a horizon search technique. However, othersuitable graph search techniques may be used. A precomputed static costis used to estimate a utility cost for a given ego position at the endof the planning time horizon. In one embodiment, the trajectory planningmodule 106 includes the graph search technique that the ego vehicle 114may follow through the environment during the configurable time horizonT. In another embodiment, the graph search technique is stored on theaction planning module 104. The generated trajectory is then stored inthe computer readable medium 110. Further details on action planningusing iterative action search full environment prediction will bedescribed below.

FIGS. 4A-4D are views of the autonomous vehicle 114 on a two-lane road200 traveling autonomously in proximity to a plurality of neighboringvehicles 160, 162 according to a described embodiment of the disclosure.As depicted in FIG. 4A, the vehicles 160 is currently following the samepotential path 202 on the left lane of the two-lane road 200. Theautonomous vehicle 114 is currently positioned in the right lane of thetwo-lane road 200, with the left and right lanes separated by a dividingline 204. The autonomous vehicle 114 or “an ego vehicle” describedherein starts in a follow lane action and enumerates all possibleactions it can switch to at the next time step. In one aspect, asdepicted in FIG. 4A, the ego vehicle 104 continues in the follow laneaction and follows the same potential path 208 on the right lane of thetwo-lane road 200. In another aspect, the autonomous driving system 100of the ego vehicle 104 may switch from the follow lane action to a lanechange action and a planned path 210 for the ego vehicle 104 is depictedin FIG. 4B as crossing over the dividing line 204 to approach thepotential path 202 of the neighboring 160 if the lane change action hasa lower initial cost. The action planning module 104 having horizonsearch programmed therein explores switching from the follow lane actionto the lane change action. In one embodiment, an iterativefull-environment prediction is used which allows the horizon search toevaluate reactions and interactions with other vehicles 160, 102 withoutexplicitly searching over the possible actions of other vehicles 160,162. The iterative full-environment prediction keeps the horizon searchtractable. Each time a new action is evaluated in the horizon graphsearch, a prediction process is performed that estimates futureenvironment of the ego vehicle 114 at the next timestep assuming the egovehicle 114 follows the selected action, e.g. a follow lane action, alane change action, or an abort lane change action. This predictionprocess includes generating a possible trajectory for the ego vehicle114 and then predicting the motion and reactions of each dynamicobstacle in the environment of the ego vehicle 114 based on its currentlocal traffic context. The prediction process further includesgenerating a prediction of how other vehicles may react to their currenttraffic situation and iteratively over short timesteps. The horizongraph search is able to model interactions between the ego vehicle 114and other traffic participants without requiring to do any activeplanning for the other vehicles 160, 162. The iterative passiveprediction is much less expensive and scales much better than attemptingto perform joint active planning for the ego vehicle 114 and othertraffic participants.

In one aspect, the ego vehicle 104 can either continue in the lanechange action or switch from the lane change action to an abort lanechange action. As depicted in FIG. 4C, the ego vehicle 104 proceeds tocontinue in the lane change action by entering the potential path 202 ofthe neighboring vehicle 160 and the planned path 210 thereby merges tothe potential path 202. If either the ego vehicle 104 decides not toswitch lanes, or not likely able to change lanes due to on-comingneighboring vehicle 160 approaches the potential path 202, as depictedin FIG. 4D, the autonomous driving system 100 of the ego vehicle 104switches from the lane change action to the abort lane change action. Aplanned path 212 for the ego vehicle 104 is shown as crossing over thedividing line 204 to approach the left lane of the two-lane road 200.

Now returning to FIG. 4B, at timestep t1, the ego vehicle 114 generatesa lane change trajectory but the other vehicle 160 in the left lane hasnot yet reacted. At timestep t2, if the ego vehicle either continueswith the lane change behavior as illustrated in FIG. 4C or switches toan abort lane change action as shown in FIG. 4D, the ego vehicletrajectory takes the ego vehicle 114 partially into the left lane,causing a reaction for the left-lane vehicle 160. In some embodiments,restricting the action search to only evaluate switching behaviors at asmall subset of timesteps, effectively decoupling the iterativeprediction time resolution from the action search time resolution. Thisgreatly improves computational efficiency and allows for the evaluationof longer planning horizons, which generates more intelligent actiondecisions. The limited horizon search though the tree of possible egoactions for intelligent automated driving decision making provides asignificant decision making performance improvement. In one example, theego vehicle 114 explicitly evaluates the expected utility of its actionsat future timesteps. In another example, the ego vehicle 114 understandsincrementally predicts the actions of other agents based on its ownanticipated future state. The ego vehicle 114 explicitly evaluates andcompares different possible timings of each action transition tomaximize safety, comfort, and progress towards its goal. Iterativeprediction allows the action search to understand how other dynamicobstacles react to and interact with the ego vehicle 114 without theneed to perform joint anticipatory planning for each dynamic obstacle inthe environment. Restricted branch points decouple the action decisiontime resolution from the iterative prediction resolution, allowing moreefficient exploration of longer planning horizons.

The embodiments described above have been shown by way of example, andit should be understood that these embodiments may be susceptible tovarious modifications and alternative forms. It should be furtherunderstood that the claims are not intended to be limited to theparticular forms disclosed, but rather to cover all modifications,equivalents, and alternatives falling with the sprit and scope of thisdisclosure.

Embodiments within the scope of the disclosure may also includenon-transitory computer-readable storage media or machine-readablemedium for carrying or having computer-executable instructions or datastructures stored thereon. Such non-transitory computer-readable storagemedia or machine-readable medium may be any available media that can beaccessed by a general purpose or special purpose computer. By way ofexample, and not limitation, such non-transitory computer-readablestorage media or machine-readable medium can comprise RAM, ROM, EEPROM,CD-ROM or other optical disk storage, magnetic disk storage or othermagnetic storage devices, or any other medium which can be used to carryor store desired program code means in the form of computer-executableinstructions or data structures. Combinations of the above should alsobe included within the scope of the non-transitory computer-readablestorage media or machine-readable medium.

Embodiments may also be practiced in distributed computing environmentswhere tasks are performed by local and remote processing devices thatare linked (either by hardwired links, wireless links, or by acombination thereof) through a communications network.

Computer-executable instructions include, for example, instructions anddata which cause a general purpose computer, special purpose computer,or special purpose processing device to perform a certain function orgroup of functions. Computer-executable instructions also includeprogram modules that are executed by computers in stand-alone or networkenvironments. Generally, program modules include routines, programs,objects, components, and data structures, etc. that perform particulartasks or implement particular abstract data types. Computer-executableinstructions, associated data structures, and program modules representexamples of the program code means for executing steps of the methodsdisclosed herein. The particular sequence of such executableinstructions or associated data structures represents examples ofcorresponding acts for implementing the functions described in suchsteps.

While the patent has been described with reference to variousembodiments, it will be understood that these embodiments areillustrative and that the scope of the disclosure is not limited tothem. Many variations, modifications, additions, and improvements arepossible. More generally, embodiments in accordance with the patent havebeen described in the context or particular embodiments. Functionalitymay be separated or combined in blocks differently in variousembodiments of the disclosure or described with different terminology.These and other variations, modifications, additions, and improvementsmay fall within the scope of the disclosure as defined in the claimsthat follow.

What is claimed is:
 1. A non-transitory computer-readable storage mediumhaving stored thereon a computer program for evaluating reactions andinteractions with one or more vehicles, the computer program comprisinga routine of set instructions for causing the machine to perform:searching through a tree of possible action sequence combinations for anego vehicle; and capturing reactions and interactions with one or morevehicles.
 2. The non-transitory computer-readable storage medium ofclaim 1 wherein the capturing, by an iterative full environmentprediction, reactions and interactions with one or more vehicles.
 3. Amethod comprising: estimating, by one or more processors, futureenvironment of an autonomous vehicle; generating, by the one or moreprocessors, a possible trajectory for the autonomous vehicles;predicting, by the one or more processors, motion and reactions of eachdynamic obstacle in the future environment of the autonomous vehiclebased on current local traffic context; and generating, by the one ormore processors, a prediction iteratively over timesteps.
 4. The methodof claim 3 further comprising: decoupling, by the one or moreprocessors, an action decision time resolution from an iterativeprediction resolution.
 5. A system for an autonomous vehicle comprising:one or more processors; and one or more non-transitory computer-readablestorage medium having stored thereon a computer program used by the oneor more processors, wherein the computer program causes the one or moreprocessors to: estimate future environment of an autonomous vehicle;generate a possible trajectory for the autonomous vehicles; predictmotion and reactions of each dynamic obstacle in the future environmentof the autonomous vehicle based on current local traffic context; andgenerate a prediction iteratively over timesteps.
 6. The system of claim5 wherein the computer program further causes the one or more processorsto: decouple an action decision time resolution from an iterativeprediction resolution.